The following report contains model performance metrics for the NY City Hourly Probabilistic Residential Energy Demand Forecasting Pipeline.
Model performance was evaluated on both long-term and day-ahead forecasts. Evaluation was conducted using a holdout dataset of hourly energy
demand values between 2023-09-26 and 2024-04-27.
Long-Term Forecasting Performance
The following table contains performance metrics for the forecasting model compared with a yearly moving average baseline model.
|
MSE |
Weighted MSE |
MAE |
MAPE |
Wilcoxon Test p-value |
| Forecasting Pipeline |
128742.60 |
126349.98 |
270.23 |
0.05 |
NaN |
| Baseline Yearly MA |
604819.81 |
604819.81 |
626.46 |
0.13 |
<0.01 |
The following Plotly Figure helps to contextualize the forecasting model's performance by showing its predictions along with the actual energy demand
values. It also presents the 95% confidence interval bounds estimated by the forecasting model.
Day-Ahead Forecasting Performance
The following table contains performance metrics for the forecasting model compared with a yearly moving average baseline model.
|
MSE |
Weighted MSE |
MAE |
MAPE |
Wilcoxon Test p-value |
| Forecasting Pipeline |
107198.99 |
107187.48 |
247.38 |
0.05 |
NaN |
| Baseline Moving Avg |
353146.56 |
353146.56 |
515.37 |
0.10 |
<0.01 |
| EIA Forecasts |
113145.06 |
NaN |
291.30 |
0.06 |
<0.01 |
The following Plotly Figure helps to contextualize the forecasting model's performance by showing its predictions along with the actual energy demand
values. It also presents the 95% confidence interval bounds estimated by the forecasting model.
The following figure shows a permutation feature importance analysis.